Host computers, including servers and client computers, are typically interconnected to form computer networks. A computer network, and more generally a communications network, is a group of devices or network entities that are interconnected by one or more segments of transmission media on which communications are exchanged between those network entities. The communications can be transmitted electrically, including wireless links, or optically. The computer networks typically further comprise separate network communications devices, such as routers, switches, bridges, and hubs, for transmitting and relaying the communications between the network entities through the network's mesh.
Computer networks are typically classified by their size or by the type of entity that owns the network. Often, business organizations maintain large computer networks. These computer networks are referred to as enterprise networks. Enterprise networks are typically connected to other enterprise networks or home networks via service provider and public networks.
At the enterprise, service provider, and public network scale, network management systems are used to monitor networks. These systems can exist as stand-alone, dedicated systems or be embedded in network communications devices such as routers and switches. One specific example is NetFlow technology offered by Cisco Systems. Other tools include special-purpose systems, such as firewalls and other network security devices, that are typically used to manage the communications at boundaries between the networks.
One source of information for monitoring networks is flow information. This is defined as “a unidirectional sequence of packets with some common properties that pass through a network device.” Internet Engineering Task Force, RFC 3954. Flow records are often generated by the network devices. These are often digested information concerning individual network flows or groups of network flows sharing some common characteristic(s). The flow records often include, for example, internet protocol (IP) addresses, packet and byte counts, timestamps, Type of Service (ToS), application ports, input and output interfaces, to list a few examples. This information is available from Netflow technology, for example. Generally, computer network devices that generate flow records include, for example, routers, switches, firewalls, and hubs. In other examples, packet scanners/analyzers (e.g. Arbor Networks PEAKFLOW® threat management system (TMS)) are used. Flows may be collected and exported for analysis. Flow analysis is a central component of large-scale network management and service systems.
Network management systems allow the network administrators to apply policies. Policies are typically used to govern or dictate how entities are allowed to communicate over the network, generally called security policies. These policies can be applied to entities individually, by setting operating parameters of devices separately. Policy-based management systems have simplified configuration of devices by allowing administrators to define a policy and apply this policy across groups of network entities, generally.
A policy is a collection of rules. A rule, for example, can be defined to govern what traffic a particular firewall ignores or prevents a given address or device from accessing a particular service or network resource. The rules can be applied by routers that decide whether to forward packets from or to a particular address.
Network policies are often defined and applied based on flow information. Moreover, many products are available that attempt to correlate flow information with other data sources to provide value-added analysis. These types of analysis tools are now a central component of administering large communication networks. Such analysis facilitates the creation of higher level policies that facilitate the management of the network.
By way of additional background, the process for abstracting the dataflow between the network entities is typically articulated in the context of the OSI (Open Systems Interconnection) model communications stack. The lowest layer 1 describes physical layer functions such as the transmission of bits over the communication medium, activation/deactivation of the physical connection, use of idle conditions, control bit generation/detection, start and stop, and zero bit insertion. These functions are requested by data link layer 2 functions, which control the transmission of packets over a logical communications link. Other data link functions include establishing/releasing logical connections, error detection, correction, and recovery, in conjunction with the delimiting of transmitted packets.
At the next higher level of abstraction is the network layer 3. Functions here include the transfer of units or packets between two transport entities. Further, at this layer, routing through the network is determined, including segmenting or combining packets into smaller and larger data units, the establishment, maintenance, and relinquishment of end-to-end logical circuits, and the detection and recovery from errors. Network management activities often take place at the network layer and data link layer.
Then, transport layer 4 functions handle the transmission of complete messages between network entities. At this layer, sessions between the network entities are established and then taken down. This layer ensures the correct sequence of packets, partition, and combination of messages into packets, and the control of data flow to avoid network overload.
The session layer 5 organizes and synchronizes the dialog that takes place between applications running on network entities. This provides a one-to-one correspondence between a session connection and a presentation connection at a given time. It provides for session continuity, even when transport connections may fail.
Finally, at the two highest levels of abstraction, layers 6 and 7, the presentation layer provides independence from differences between data presentations, such as encryption, by translating from application to network format, and back. The application layers support application and end user processes. However, user authentication and privacy are also considered and any constraints on data syntax are identified. At this layer, communication is application-specific.